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1. BACKGROUND AND OBJECTIVES 


An evaluation of the labeling performance in Phase III and in the Transition 
Year (TY) of the Large Area Crop Inventory Experiment (LACIE) identified 
spring small-grain (SSG) omission errors as a major problem in obtaining 
unbiased acreage estimates. One of the prime causes for the large omission- 
error rate was the variability in signatures for small grains. Frequently, 
these signatures were considered unusual because of early or late emergence 
and development (ref. 1). 

The signature for small grains can be represented as a profile of 
Landsat spectral value versus time (acquisition date) for each of the four 
channels of data. Early or late emergence and development are expected to 
shift this profile earlier or later (to the left or right) along the time 
axis. A method for estimating both the shift due to late or early emergence 
as well as the similarity of shape of the profile to that of SSG's has been 
developed by G. D. Badhwar (ref. 2). This method requires at least five 
Landsat acquisitions (though not necessarily on different dates) and a single 
interpreted SSG training field. A hypothetical profile for SSG's showing the 
amount of shift along the time-axis which is referred to as Tau (x) may be 
seen in figure 1-1. How well the profile and the picture element (pixel) 
selected for labeling fits the model profile of the SSG training field is 
designated as the Chi-squared "goodness-of-fi t" (x )• 

The purpose of this study was to determine the feasibility of using the pro- 
file similarity concept to reduce the analyst SSG omission-error rates. However, 
because of the acquisition history requirements of the algorithm, the 
utility of this concept would apply primarily to at-harvest or near-harvest 
crop estimations. 
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2. APPROACH 


Omission errors are caused by small-grain pixels which are mistaken by the 
analyst to be nongrain. Conceptually, the profiles of these mislabeled 
pixels should be similar to those of SSG, A measure of the degree of simi- 
larity, after adjustment for late or early emergence and development, is the 
Chi-squared value. Thus, the Chi-squared statistic can possibly be used as 
a means of detecting omission errors among the pixels labeled nongrain by the 
analysts. The smaller the Chi-squared value in any given Landsat channel, the 
better is the fit to the profile. This implies that if the Chi-squared value 
is low, there is a high probability that a given pixel is a small-grain pixel 
with the proviso that the pixel must be pure (A pure pixel is one that stays 
in the same category on all acquisitions used in processing.). Scale factors 
from one channel to the next must be considered if all four channels are to be 
utilized in obtaining a "correlation" between the Chi-squared value and the 
crop identity. Another group of dots needed to be included to provide an 
indication of "false alarms" that might be caused using this approach. These 
are the nonsrnal 1-grain dots that were correctly labeled. 

In this study, the Chi-squared values of all the pure dots that were labeled 
nonsmall grain were ranked one at a time; the ranks for each dot were summed, 
and the sums of the individual channel ranks were then ranked. An example of 
this procedure is shown in table 2-1. This procedure compensates for scale 
factors, reduces the dimensionality of the data, and gives equal weight to 
all of the channels . 
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3. DATA SET DESCRIPTION 


A data set of six LACIE/TY blind sites containing SSG's was obtained. These 
six segments met the following criteria that were required by this study: 

a. LACIE/TY ground truth for dot labels 

b. An acquisition history consisting of a minimum of four acquisitions 
excluding pre-emergence or harvest acquisitions (Channel 1 may be dupli- 
cated to meet the required minimum of five for the algorithm.) 

c. Appreciable amounts of SSG's (spring wheat, oats, and barley) 

d. Analyst omission errors from LACIE/TY operational processing 

The segments, acquisition histories, and the atmospheric conditions for these 
acquisitions are presented in table 3-1. For each of the six segments, three 
SSG training fields were selected based on ground-truth labels, size (minimum 
20 pixels), and homogeneity. Each of these three fields were then processed 
through the program yielding three sets of data for each segment that the 
ranking system processed. One optimum field was then selected for each seg- 
ment based on the means and minimum standard deviations of the data in the 
four channels and the minimum Chi-squared values in each of the four channels. 
These values are shown in table 3-2. 

Of the thirty-six segments in North Dakota, Minnesota, Montana, and South 
Dakota, approximately seven segments met all the criteria required by the 
program. The remainder were rejected primarily because of poor acquisition 
histories (insufficient acquisitions or acquisitions that did not fall between 
emergence and ripe-crop conditions). One segment (segment 1542 in Roosevelt, 
Montana) of the six originally selected was later dropped from the analysis 
for lack of analyst omission errors (i.e., after the border/edge dots were 
eliminated manually, there were not enough pure or interior SSG dots to 
warrant inclusion of this segment in the data set). 


TABLE 3-1. -THE SEGMENTS, ACQUISITION HISTORIES, AND QUALITY OF THE ACQUISITIONS 






State 

County 

Acquisitions: 

available 

North Dakota 

Stutsman 

81 35 
8154 

8207 

8217 

8226 

8243 

8270 

North Dakota 

Burleigh 

81 36 
8154 
8191 
8208 
8217 

North Dakota 

) 

j 

1 

Burke 

8120 

8156 

8174 

8211 

8219 

8228 

8264 

8116 

Minnesota 

Norman 

j 

8169 

8187 

8196 

8206 

8223 

8232 

North Dakota 

Hettinger 

! 

8156 

8191 

8209 

8218 

8228 

8236 

8246 




Acquisitions 

Quality of 

used/duplicated 

acquisitions 

8154 

~ 300 pixels of 
clouds/shadow 

8207 


8217 


8226 

~ 600 pixels 
of haze 

8243 

Haze 

8154 


8191 

Si 200 pixels of 

8191 

popcorn clouds 

8208 


8217 


8156 


8174 


8211 

S 300 pixel s 
of clouds 

8219 


8228 


8169 

Light haze 

8196 


8206 


8223 


8232 


8191 

= 600 pixels of 
clouds/shadow 

8209 


8218 


8228 


8236 


























TABLE 3-2. MEANS, STANDARD DEVIATIONS, AND CHI-SQUARED VALUES OF TRAINING FIELDS 
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4. TRAINING FIELD SELECTION 

For the algorithm to perform optimally, the training fields selected must 
contain a min . /V of 20 pixels, must be homogeneous, must be in an emerged 
stage by the first acquisition date, and must not be harvested by the last 
acquisition date (ref. 3). Of the five training fields (one per segment) 
finally selected and used, only one field (segment 1653) met the subjective 
optimum criteria described above. For the remaining four segments, little 
could be done to improve the training fields selected because of one or more 
of the following reasons: 

a. Inadequate number of sufficiently large training fields in segments 

b. Inadequate number of homogeneous training fields 

c. Inadequate number of emerged SSG training fields from which to choose. 


If 


5. RESULTS 


In table 5-1, the ranks have been divided into four equal portions over the 
entire data range for each segment. The numbers in the quartile columns 
represent the percentages of small grains and nonsmall grains (according to 
ground truth) that fall into each quartile. Optimally, the SSG dots should 
fall within the first quartile with nonsmall grains occupying the other 
quartiles. (The SSG dots should be within the first 25 percentile of the 
dots and have the lowest Chi-squared values.) It is evident that the SSG's 
are spread over the range of the ranks except in segment 1653 which has the 
majority of small-grain dots in the first quartile. 

In table 5-2, only small-grain dots are considered. The numbers represent 
the percentage of small -grain dots compared to the total number of small -grain 
■',i,.ts falling in each quartile of the range. Again, it can be seen that small - 
grain dots are spread throughout the quartiles except in segment 1653 where 
the majority of small-grain dots are concentrated in the first quartile. 

Generally, channel 1 rankings (0.50 to 0.60) displayed the greatest variability. 
The ranks appeared randomly and showed a poor relationship to their ground- 
truth labels. Since this channel is most susceptible to atmospheric effects, 
it could be inferred that the ranks in channel 1 do not contribute appreciably 
to the ability to differentiate between small grains and nonsmall grains using 
this approach. 

Casual observation also indicates that the ranks in channel 3 and channel 4 
display a better relationship to the ground-truth labels than do channel 1 
and channel 2. Clear patterns were not seen in the rankings of the dots using 
this procedure except in segment 1653. The reason for the relative success 
in segment 1653 is found in the good acquisition history and the relatively 
similar profile to the ideal SSG profile. Segment 1653 had postemergence 
and preharvest acquisitions for most small-grain fields in the scene and for 
the training field. 
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TABLE 5-2.- PERCENTAGE OF OMISSION-ERROR DOTS (SSG ONLY) 
FALLING WITHIN QUARTILES OF DATA RANGE 


Segment 

number 

1 st 

Quart! le 

2nd 

Quartile 

3rd 

Quartile 

4th 

Quartile 

Total omi 
error 

1636 

29.4 

23.5 

20.6 

26.5 

34 

1653 

69.2 

23.1 

0 

7.7 

13 

1394 

55.3 

18.4 

26.3 

0 

19 

1825 

30.0 

20.0 

20.0 

30.0 

10 

1650 

37.5 

62.5 

0 

0 

8 
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6. OBSERVATIONS ON THE INDIVIDUAL SEGMENTS 


The segments, the acquisitions that were utilized, the growth stages for the 
training fields, the colors of the training fields, the relationships between 
the training field conditions and the rest of the SSG fields in the scene and 
atmospheric conditions are shown in table 6-1. All these factors are per- 
tinent to the study in that they explain why certain segments failed to per- 
form as expected. Figure 6-1 represents the shapes of the ideal SSG training 
field profiles in each of the four Landsat channels. Figures 6-2, 6-3, 6-4, 
6-5, and 6-6 are the profiles of the training fields for each segment. Com- 
parison of the. ideal profile with the actual profiles also provides an insight 
as to why some segments failed to perform. 

6.1 SEGMENT 1636, STUTSMAN, NORTH DAKOTA 

The acquisitions utilized were June, 3, July 26, August 5, August 14, and 
August 31. The SSG crop stages for the training fields corresponding to 
these dates were planting, tillering, heading, turning, and ripening, 
respectively. 

Analysis of the segment also shows that the other spring grain fields in the 
scene are generally in the same stages of growth as the training field. The 
shapes of the model training-field profile for channels 1 and 2, shown in 
figures 6-2a and 6-2b poorly match the idealized situation in figure 6-1. 
Channels 3 and 4 (figures 6-2c and 6-2d) more closely approach the ideal case. 
The channel 1 profile (figure 6-2a) also shows a peculiar deflection at acqu- 
isition date 8217 (August 5). The imagery shows no evidence of anything 
unusual in regard to atmospheric haze or clouds. This deflection is absent 
from channels 3 and 4 (figures 6-2c and 6-2d) and is less pronounced in 
channel 2 (figure 6-2b). 

6.2 SEGMENT 1653, BURLEIGH, NORTH DAKOTA 

The acquisitions utilized were June 3, July 10, July 27, and August 5. The 
SSG crop stages for the training field corresponding to these dates are 
slightly emerging, tillering, turning, and ripening. Analysis of the segment 
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also shows that the SS6 fields in the scene are generally in the same stages 
of growth as the training field. The exception is on the acquisition of 
June 3, which shows that the training field is behind with respect to the 
other SSG fields in the scene. The shapes of the model training field very 
closely approach the ideal situation (figures 6-3a, 6-3b, 6-3c, and 6-3d). 

6.3 SEGMENT 1394, BURKE, NORTH DAKOTA 

The acquisitions utilized were June 5, June 23, July 30, August 7, and 
August 16. The SSG crop stages for the training field corresponding to these 
dates are slightly emerging, slightly emerging, tillering, early turning, and 
late turning. Except for the acquisition of July 11, the training field is 
behind the other fields in the segment. The model profiles (figures 6-4a, 
6-4b, 6-4c, and 6-4d) for the training field do not resemble the idealized 
profile. 

6.4 SEGMENT 1825, NORMAN, MINNESOTA 

The acquisitions utilized were June 18, July 15, July 25, August 11, and 
August 20. The crop stages for the training field corresponding to these 
dates are tillering, heading, turning, ripening, and ripening. Analysis of 
the segment indicates that SSG fields in the scene are generally in the same 
stages of growth as the training field, except the August 20 acquisition which 
shows barley and other SSG's already harvested. The model profiles (fig- 
ures 6-5a, 6-5b, 6-5c, and 6-5d) somewhat resemble the idealized profiles. 

6.5 SEGMENT 1650, HETTINGER, NORTH DAKOTA 

The acquisitions utilized were July 10, July 28, August 6, August 16, and 
August 24. The SSG crop stages for the training field corresponding to these 
dates are postemergence, heading, heading, turning, and ripening. Analysis 
of the segment also shows that SSG fields are generally in the same stages of 
growth as the training field. The model profiles (figures 6-6a, 6-6b, 6-6c, 
and 6-6d) do not resemble the idealized profiles. 
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TABLE 6-1.- DATA ON TRAINING FIELDS AND OTHER SSG FIELDS IN SCENE 
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Figure 6-1.- Idealized SSG profile. 
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Figure 6-2a Segment 1636 spring wheat, channel 1. 
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Figure 6-2b.- Segment 1636 spring wheat, channel 2. 
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Figure 6-2c.- Segment 1636 spring wheat, channel 3. 
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Figure 6-2d.— Segment 1636 spring wheat, channel 4. 
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Figure 6-3c.- Segment 1653 spring wheat, channel 3. 
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Figure 6-4a.— Segment 1394 spring wheat, channel 1. 
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Figure 6-4b.- Segment 1394 spring wheat, channel 2. 


PLOT riEPNS AND +/- ONE STAND DEO 
3 ,1433 3 3330 0,3301 1.3183 

? 1436 3 S793 0,5357 3,8384 

FITTED VALUES ARE 37,95 44,63 53,64 53,16 53,33 

I CHISQR- 0.3474E+01 



30 00 I 


0 00 





130,00 153,00 174,00 196,00 318.00 


CALENDAR DAY 


Figure 6-4c.- Segment 1394 spring wheat, channel 3. 
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Figure 6-4d,— Segment 1394 spring wheat, channel 4. 
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Figure 6-5a.- Segment 1825 spring wheat, channel 1. 
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Figure 6-5b.— Segment 1825 spring wheat, channel 2. 
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Figure 6-5c.- Segment 1825 spring wheat, channel 3 






lil-’PRODUClIilLITY OP THE 
ORIGINAL PAGE IS POOR 


4 6013 


PLOT MEANS AND ONE STAND DEU 
- 33 . 5760 - 3 . 8030 1.6511 



CALENDAR DAY 


Figure 6-6a Segment 1650 spring oats, channel 1 
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Figure 6-6b.- Segment 1650 spring oats, channel 2. 
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Figure 6-6c.- Segment 1650 spring oats, channel 3. 
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Figure 6-6d.— Segment 1650 spring oats, channel 4. 


7. CONCLUSIONS AND ISSUES 


7.1 COMPATIBILITY WITH CURRENT OPERATIONAL PROCEDURES 

Current direct wheat (spring wheat and barley separation) classification 
procedures (ref. 4) require estimations in real time throughout the growing 
season using up to four acquisitions ranging from pre-emergence to postharvest. 
It is quite apparent that, given the acquisition history requirements of the 
profile similarity algorithm, the utility of this concept applies only to 
near- or at-harvest estimation procedures. 

7.2 ACQUISITION HISTORY REQUIREMENTS 

The Badhwar procedure appears to be highl'y dependent upon the optimum acqui- 
sition history for the segments. In all, 36 segments were screened for 
suitability and only 7 were found to meet the criteria. In general, where 
the segments were found to have a sufficient number of acquisitions for the 
algorithm, few LACIE/TY analyst omission errors (for pure dots) could be found. 
Segment 1542 in Roosevelt, Montana, is a case in point. Although this segment 
had a good acquisition history and a clearly defined field pattern, it later 
had to be dropped from the data set because of an insufficiency of analyst 
omission errors applied to pure dots. LACIE/TY data indicated that most of 
the omission errors were caused by border/edge conditions. 

7.3 TRAINING FIELD REQUIREMENTS 

In addition to acquisition history, training field selection appears tg be 
one of the most crucial aspects for optimum performance using this approach. 

For the algorithm to perform, the training field requires a minimum of 
20 pixels, which also should be homogeneous; must be an interior field (to 
eliminate misregistration effects); and the crops in the field should have 
emerged by the first acquisition date and not be harvested by the last 
acquisition date (ref. 3). Of the five training fields selected, only one 
field (segment 1653) met the subjective optimum criteria that has been set. 




Training fields could be improved by a process of trial an-.} error. However, 
at least three outcomes may result in trying to improve che selection: 

a. There may not be an adequate number of large enough small-grain training 
fields in the segment because of the interior field requirement. 

b. There may not be an adequate number of homogeneous small -grain training 
fields in the segment. 

c. There may not be an adequate number of emerged small -grain training fields 
in the segment. Addi tionally, in order to select a training field that 
had emerged by the first acquisition, one acquisition at the front would 

I have to be dropped. This leads to at least three more possible outcomes: 

1. That same field would in all likelihood be harvested by the last 
acquisition date, requiring elimination of the last acquisition date 

} as well . 

2. The other spring-grain fields in the segment might also be harvested. 

3. There may not be a sufficient number of acquisitions to establish a 
model curve for spring grains after the last acquisition was dropped 
as a result of 1 and 2 above. 

7.4 DOT PURITY REQUIREMENTS 

, J, 

1 “ In this study, the border/edge dots were eliminated manually. For this pro- 
cedure to perform even quasi -operationally, a mechanism must be developed to ! 

identify the border/edge dots and subsequently eliminate them. •' 

ii 

I 7.5 APPL ICATION TO TYPE OF ERROR 

I • 

I One of the purposes of this study was to determine if the Chi -squared value 

if (or rank) could serve as a flag for possible small -grain omission errors. To 

I accomplish this, another group of dots was included to provide an indication 

I of "false alarms" that might be caused using this approach. This leads to a 

( procedure whereby all dots labeled nonsmall grain by the analyst need to be 
examined for omission errors. However, there is a possibility that all these 



dots were correctly labeled nonsmall grain in the first place. The results 
indicate that there is a high probability that nonsinall -grain dots could be 
falsely flagged as SSG. 
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8. RECOMMENDATIONS 


Further research needs to be conducted in the following areas prior to the 

development of analysis procedures which might use thi^ technique. 

a. The channels or combination of channels that are most appropriate for 
use in measuring profile similarity should be determined. 

b. The model chosen by Badhwar requires a nonlinear curve fitting. This is 
an iterative procedure where initial parameter estimates must be provided 
and convergence is not guaranteed. A study of alternative model forms 
that can use 1 inear-1 east-squares should be investigated; this would remove 
the technique's dependence upon the initial parameter estimates. 

c. The sensitivity of the technique to the training field selection should 

be measured. The need is to better understand the requirements for train- 
ing field selection. This understanding includes homogeneity of emergence 
and development and similarity to the usual profile for the crops of 
interest. 

d. The relative frequency at which the severe acquisition history constraints 
are met for the crop of interest should be estimated. This will establish 
how often the technique can be applied to various crops of interest. 

In addition, there are other applications of the technique other than omission 

errors which should be evaluated. They are: 

a. The detection of commission errors 

A 

b. Spectral adjustment for early or late development to enhance the barley/ 
other SSG's separation 

c. Study of crop calendar planting date distributions 
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